AdaMoW: Multimodal Sentiment Analysis Based on Adaptive Modality-Specific Weight Fusion Network

Multimodal sentiment analysis (MSA) is a crucial task in the field of natural language processing (NLP), with wide range applications. This paper proposes an adaptive modality-specific weight fusion network (AdaMoW) to address issues process multimodal data fusion. Specifically, we use different calculation methods at various stages model. In model training stage, diverse weights are assigned modalities by calculating correlation between single-modal prediction value and real labels, weight-mapping designed learn this “data-weight” mapping relationship. testing verification phase model, trained used obtain modalities. addition, order optimize data, generator, which reversely generates unimodal feature vector through vector, compares it original extraction obtained after extraction. The modal vectors compared optimized, so that results can maintain uniqueness modality while obtaining interaction information. AdaMoW verified on two benchmark MSA datasets CMU-MOSI CMU-MOSEI. experimental show effectiveness surpasses previous baseline achieves state-of-the-art results.